Efficient Machine Learning in Hardware

Lec­turer	Oliver Bring­mann HeadOliver Bring­mann
Lec­ture	21.04.2022-28.07.2022 Don­ner­stag 12ct-14
In­struc­tors	Alexan­der Jung Re­searcherAlexan­der Jung Adrian Frischknecht AlumniAdrian Frischknecht Christoph Gerum Re­searcherChristoph Gerum Evge­nia Rusak Re­searcherEvge­nia Rusak Kon­stan­tin Lübeck Re­searcherKon­stan­tin Lübeck Paul Palom­ero Bernardo Re­searcherPaul Palom­ero Bernardo Simon Garhofer Re­searcherSimon Garhofer Moritz Reiber Re­searcherMoritz Reiber
Amount	2 SWS / 3 LP
Type of course	Lec­ture (3 LP)
Course ID	ML4420
Entry in course cat­a­log	Alma
Learn­ing Plat­form	Ilias

Topic

The re­cent break­throughs in using deep neural net­works for a large va­ri­ety of ma­chine learn­ing ap­pli­ca­tions have been strongly in­flu­enced by the avail­abil­ity of high per­for­mance com­put­ing plat­forms. In con­trast to its bi­o­log­i­cal ori­gin, how­ever, high per­for­mance of ar­ti­fi­cial neural net­works crit­i­cally re­lies on much higher en­ergy de­mands. While the av­er­age en­ergy con­sump­tion of the en­tire human brain is com­pa­ra­ble to that of a Lap­top com­puter (i.e. 20W), ar­ti­fi­cial in­tel­li­gence often re­sorts to large HPCs with sev­eral or­ders of mag­ni­tude higher en­ergy de­mand. This lec­ture will dis­cuss this prob­lem and show so­lu­tion how to build en­ergy and re­source ef­fi­cient ar­chi­tec­tures for ma­chine learn­ing in hard­ware. In this con­text, the fol­low­ing top­ics will be ad­dressed:

• Hard­ware ar­chi­tec­tures for ma­chine learn­ing: GPUs, FPGAs, over­lay ar­chi­tec­tures, SIMD ar­chi­tec­tures, do­main-spe­cific ar­chi­tec­tures, cus­tom ac­cel­er­a­tors, in/near mem­ory com­put­ing, ar­chi­tec­tures for train­ing vs. ar­chi­tec­tures for in­fer­ence
• En­ergy-ef­fi­cient ma­chine learn­ing
• Op­ti­mized map­ping of deep neural net­works to hard­ware and pipelin­ing tech­niques
• Word length op­ti­miza­tion (bi­nary, ternary, in­te­ger, float­ing point)
• Scal­able ap­pli­ca­tion spe­cific ar­chi­tec­tures
• New switch­ing de­vices to im­ple­ment neural net­works (Mem­ris­tors, PCM)
• Neu­ro­mor­phic com­put­ing

Stu­dents gain in-depth knowl­edge about the chal­lenges as­so­ci­ated with en­ergy-ef­fi­cient ma­chine learn­ing hard­ware and re­spec­tive state-of-the-art so­lu­tions. Dif­fer­ent hard­ware ar­chi­tec­tures will be com­pared re­gard­ing the trade-off be­tween their en­ergy con­sump­tion, com­plex­ity, com­pu­ta­tional speed and the speci­ficity of their ap­plic­a­bil­ity.

The main goals of the course are learn­ing what kinds of hard­ware ar­chi­tec­tures are used for ma­chine learn­ing, un­der­stand­ing the rea­sons why a par­tic­u­lar ar­chi­tec­ture is suit­able for a par­tic­u­lar ap­pli­ca­tion and how to ef­fi­ciently im­ple­ment ma­chine learn­ing al­go­rithms in hard­ware.

Lit­er­a­ture

• Sze, Vivi­enne, et al. “Ef­fi­cient pro­cess­ing of deep neural net­works.” Syn­the­sis Lec­tures on Com­puter Ar­chi­tec­ture 15.2 (2020): 1-341.